Skip to content

[ExecuTorch][WebGPU] Add permute_copy + IntList graph support (aten.permute_copy.default)#20396

Merged
JulianCloudNTH merged 6 commits into
gh/JulianCloudNTH/45/basefrom
gh/JulianCloudNTH/45/head
Jun 26, 2026
Merged

[ExecuTorch][WebGPU] Add permute_copy + IntList graph support (aten.permute_copy.default)#20396
JulianCloudNTH merged 6 commits into
gh/JulianCloudNTH/45/basefrom
gh/JulianCloudNTH/45/head

Conversation

@JulianCloudNTH

@JulianCloudNTH JulianCloudNTH commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Stack from ghstack (oldest at bottom):

Adds aten.permute_copy.default (a coordinate-reorder gather) to the WebGPU delegate, and the IntList graph value type it needs to read its dims argument.

Composition:

  • runtime/WebGPUGraph.{h,cpp} — adds ValueType::IntList backed by std::vector<std::vector<int64_t>> int_lists_ + get_int_list(int); build() deserializes vkgraph::GraphTypes::IntList via value_as_IntList()->items() (int64, matching the FlatBuffer [long]); mirrors the existing scalar value plumbing.
  • runtime/ops/permute/Permute.cpp — reads the permutation via get_int_list, normalizes negative dims, validates it is a permutation of [0, ndim), builds two TensorMeta UBOs + a PermuteParams{perm: vec4<u32>} uniform, guards fp32 + rank≤4, dispatches over compute_1d_workgroup_count(out.numel) with override wg_size; releases all uniforms after the bind group.
  • runtime/ops/permute/permute.wgsl — delinearizes the output index over the contiguous output strides, reads input at in.strides[perm[d]] per dim (mirrors Vulkan permute_buffer.glsl).
  • Registers both aten.permute_copy.default and aten.permute.default to the same handler.
    @exported-using-ghexport

Differential Revision: D108793162

Differential Revision: D108793162

[ghstack-poisoned]
@pytorch-bot

pytorch-bot Bot commented Jun 18, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20396

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Unrelated Failure

As of commit 189d73f with merge base e03f777 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 18, 2026
@JulianCloudNTH

Copy link
Copy Markdown
Contributor Author

@claude review

@claude

claude Bot commented Jun 18, 2026

Copy link
Copy Markdown

Claude finished @JulianCloudNTH's task in 3m 11s —— View job


Code Review — PR #20396: permute_copy + IntList graph support

  • Read changed files
  • Review WebGPUGraph IntList plumbing
  • Review Permute.cpp op
  • Review permute.wgsl shader
  • Post review feedback

Overall this is a clean, well-scoped addition that faithfully mirrors the existing per-op pattern (Slice.cpp) and the Vulkan permute_buffer.glsl reference. The IntList plumbing matches the FlatBuffer [long] schema (schema/program.fbs:193-195, Vulkan schema.fbs:88-89) and the existing scalar-value conventions. I verified the WGSL gather math, the bounds/validation logic, and the uniform/bind-group lifetime. A few notes below — nothing blocking.

Correctness ✅

  • Gather math is correct. For output coord d, in_bufi += coord * in_strides[perm[d]] correctly maps output dim d to input dim dims[d] (PyTorch permute semantics: out.shape[d] == in.shape[dims[d]]). Output delinearization over contiguous out_meta.strides is sound since the AOT output is contiguous.
  • No division-by-zero in the shader. out_meta.strides[d] for d < ndim is always ≥1 with contiguous strides; an empty tensor (numel == 0) dispatches 0 workgroups.
  • No OOB on the fixed-size arrays. perm[]/seen[] are sized kTensorMetaMaxNdim (4) and the ndim > 4 guard (Permute.cpp:62) runs before the indexing loop. The permutation-validity check (negative-dim normalize + seen[] dup detection) is correct.
  • fp32 + rank≤4 guards are present and consistent with Slice.cpp.

Suggestions (non-blocking)

  1. Missing op test. Every other buffer op under backends/webgpu/test/ops/ (slice, squeeze, unsqueeze, select, view_copy, …) ships a test module, but there's no test/ops/permute/. Since permute has nontrivial index math, a correctness test across a few rank-2/3/4 permutations (including negative dims and identity) would be valuable and would also exercise the new IntList deserialization path. Add a permute op test →

  2. Significant boilerplate duplication with Slice.cpp. Permute.cpp:111-187 (shader module → bind-group-layout → pipeline → bind-group → dispatch → release) is ~95% identical to Slice.cpp:106-182, differing only in the params struct and WGSL constant. This is a pre-existing pattern across ops, so following it here is reasonable — but the duplication is now large enough that a small shared helper (e.g. "build a 1-storage-in / 1-storage-out / N-uniform pipeline+dispatch") would pay off across the growing op set. Out of scope for this PR; flagging for the backend maintainers.

  3. kPermuteWorkgroupSizeY / kPermuteWorkgroupSizeZ are unused (permute_wgsl.h:57-58). These appear to be emitted by the WGSL codegen for all shaders, so this matches the generated-header convention and is fine — just noting they're dead for this 1-D dispatch.

  4. Redundant rank guard. fill_tensor_meta (TensorMeta.h:42-44) already throws on ndim > 4, so the explicit check at Permute.cpp:62-64 is belt-and-suspenders. Harmless, and arguably clearer since it guards the array writes directly.

Verified

  • WGSL SHA d34f5973…00197 matches the embedded value in permute_wgsl.h:16 (generated header is in sync with the .wgsl source). ✅
  • IntList enum addition and int_lists_ resize in build() correctly mirror the other value types. ✅
    · branch gh/JulianCloudNTH/45/head

[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
JulianCloudNTH added a commit that referenced this pull request Jun 26, 2026
Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0)
(oldest at bottom):
* #20465
* #20464
* #20463
* #20435
* #20399
* #20398
* #20397
* #20396
* #20395
* #20394
* #20393
* #20392
* #20391
* #20390
* #20363
* __->__ #20362
* #20361
* #20360
* #20359



Adds `aten.select_copy.int` to the WebGPU delegate as a gather: picks a
fixed index along one dim, producing an output of rank (input rank - 1).

Composition (single dispatch):
- `select/Select.cpp` — reads `[self, dim, index, out]` (static `Int`
via `read_scalar`; throws on dynamic `SymInt`), normalizes +
bounds-checks dim/index, builds 2 `TensorMeta` UBOs + a
`SelectParams{dim,index}`, fp32 guard, 1D-dispatch over `numel`,
releases uniforms after the bind group.
- `select/select.wgsl` — seeds the input offset with `index *
in.strides[dim]`, delinearizes the output index, maps each out dim to
its in dim (shifted past the selected dim), relinearizes on input
strides.
@exported-using-ghexport

Differential Revision:
[D108793166](https://our.internmc.facebook.com/intern/diff/D108793166/)

Differential Revision:
[D108793166](https://our.internmc.facebook.com/intern/diff/D108793166)
JulianCloudNTH added a commit that referenced this pull request Jun 26, 2026
…ework) (#20363)

Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0)
(oldest at bottom):
* #20465
* #20464
* #20463
* #20435
* #20399
* #20398
* #20397
* #20396
* #20395
* #20394
* #20393
* #20392
* #20391
* #20390
* __->__ #20363
* #20362
* #20361
* #20360
* #20359



Registers `aten.select_copy.int` in the `cases.py` op-test framework: a
`_select_suite` of 4 configs (leading/middle/last dim + negative index)
that `generate_op_tests` exports and compares to a torch golden on Dawn.
Also adds `test/ops/select/test_select.py` (`SelectModule` + `CONFIGS` +
an export-delegation/eager smoke test) and the `aten.select_copy.int`
partitioner-allowlist entry in `tester.py`.
@exported-using-ghexport

Differential Revision:
[D108793161](https://our.internmc.facebook.com/intern/diff/D108793161/)

Differential Revision:
[D108793161](https://our.internmc.facebook.com/intern/diff/D108793161)
JulianCloudNTH added a commit that referenced this pull request Jun 26, 2026
Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0)
(oldest at bottom):
* #20465
* #20464
* #20463
* #20435
* #20399
* #20398
* #20397
* #20396
* #20395
* #20394
* #20393
* #20392
* #20391
* __->__ #20390
* #20363
* #20362
* #20361
* #20360
* #20359



Adds `aten.sigmoid.default` to the WebGPU delegate: element-wise
`1/(1+exp(-x))` over a flat fp32 buffer. On the Llama critical path
(`F.silu` -> `sigmoid` + `mul`).

Composition (single dispatch):
- `sigmoid/UnaryOp.cpp` — binds input (storage, read-only) + output
(storage) + a `Params{num_elements}` uniform, 1D-dispatches over
`num_elements` with `override wg_size` (clamped to the device limit);
mirrors the `add` op (uniform mapped-at-creation, released after the
bind group).
- `sigmoid/sigmoid.wgsl` — guards `idx >= num_elements` and writes the
logistic of each element.
@exported-using-ghexport

Differential Revision:
[D108793157](https://our.internmc.facebook.com/intern/diff/D108793157/)

Differential Revision:
[D108793157](https://our.internmc.facebook.com/intern/diff/D108793157)
JulianCloudNTH added a commit that referenced this pull request Jun 26, 2026
…k) (#20391)

Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0)
(oldest at bottom):
* #20465
* #20464
* #20463
* #20435
* #20399
* #20398
* #20397
* #20396
* #20395
* #20394
* #20393
* #20392
* __->__ #20391
* #20390
* #20363
* #20362
* #20361
* #20360
* #20359



Registers `aten.sigmoid.default` in the `cases.py` op-test framework: a
`_sigmoid_suite` (hard-coded shapes + a saturation case over a
`linspace(-12, 12)` input) that `generate_op_tests` exports and compares
to an fp64 torch golden on Dawn. Also adds
`test/ops/sigmoid/test_sigmoid.py` (`SigmoidModule` + `N` + `_det_input`
+ an export-delegation/eager smoke test) and the `aten.sigmoid.default`
partitioner-allowlist entry in `tester.py`.
@exported-using-ghexport

Differential Revision:
[D108793159](https://our.internmc.facebook.com/intern/diff/D108793159/)

Differential Revision:
[D108793159](https://our.internmc.facebook.com/intern/diff/D108793159)
JulianCloudNTH added a commit that referenced this pull request Jun 26, 2026
…20392)

Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0)
(oldest at bottom):
* #20465
* #20464
* #20463
* #20435
* #20399
* #20398
* #20397
* #20396
* #20395
* #20394
* #20393
* __->__ #20392
* #20391
* #20390
* #20363
* #20362
* #20361
* #20360
* #20359



Adds `aten.squeeze_copy.dims` and `aten.unsqueeze_copy.default` to the
WebGPU delegate. Both are numel-preserving shape ops; on a dense
row-major buffer backend they are the same flat copy as `view_copy` —
only the shape metadata differs (mirrors the Vulkan delegate, which
routes both through `add_view_copy_node`).

Composition (no new kernel):
- `squeeze/Squeeze.cpp` — reads `args = [self, dims, out]`, ignores the
AOT-fixed `dims`, calls `add_flat_copy(graph, in, out)` from
`runtime/ops/view_copy/view_copy.h`.
- `unsqueeze/Unsqueeze.cpp` — reads `args = [self, dim, out]`, ignores
the AOT-fixed `dim`, calls `add_flat_copy(graph, in, out)`.
@exported-using-ghexport

Differential Revision:
[D108793153](https://our.internmc.facebook.com/intern/diff/D108793153/)

Differential Revision:
[D108793153](https://our.internmc.facebook.com/intern/diff/D108793153)
JulianCloudNTH added a commit that referenced this pull request Jun 26, 2026
….py op-test framework) (#20393)

Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0)
(oldest at bottom):
* #20465
* #20464
* #20463
* #20435
* #20399
* #20398
* #20397
* #20396
* #20395
* #20394
* __->__ #20393
* #20392
* #20391
* #20390
* #20363
* #20362
* #20361
* #20360
* #20359



Registers `aten.squeeze_copy.dims` and `aten.unsqueeze_copy.default` in
the `cases.py` op-test framework: a `_squeeze_suite` of 3 configs
(squeeze leading/middle/multiple size-1 dims) and a `_unsqueeze_suite`
of 3 configs (insert dim at front/middle/last) that `generate_op_tests`
exports via `VulkanPartitioner` and compares to a torch golden on Dawn.
Also adds `test/ops/squeeze/test_squeeze.py` (`SqueezeModule` +
`CONFIGS` + `_op_delegated` smoke test),
`test/ops/unsqueeze/test_unsqueeze.py` (`UnsqueezeModule` + `CONFIGS` +
`_op_delegated` smoke test), and the two partitioner-allowlist entries
in `tester.py`.
@exported-using-ghexport

Differential Revision:
[D108793152](https://our.internmc.facebook.com/intern/diff/D108793152/)

Differential Revision:
[D108793152](https://our.internmc.facebook.com/intern/diff/D108793152)
JulianCloudNTH added a commit that referenced this pull request Jun 26, 2026
Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0)
(oldest at bottom):
* #20465
* #20464
* #20463
* #20435
* #20399
* #20398
* #20397
* #20396
* #20395
* __->__ #20394
* #20393
* #20392
* #20391
* #20390
* #20363
* #20362
* #20361
* #20360
* #20359



Adds `aten.slice_copy.Tensor` to the WebGPU delegate as a gather: each
output element is mapped back to its source input element along the
sliced dim via `start + coord * step`.

Composition (single compute dispatch):
- `runtime/ops/slice/Slice.cpp` — reads `args = [self, dim, start, end,
step, out]` via `read_scalar` (static `Int`/`Null`-sentinel default;
throws on dynamic `SymInt`); normalizes negative `dim`/`start`, clamps
`start` to `[0, in_size]`; builds two `TensorMeta` UBOs + a
`SliceParams{dim, start, step}` uniform; guards fp32; dispatches over
`compute_1d_workgroup_count(out.numel)` with `override wg_size`;
releases all uniforms after the bind group.
- `runtime/ops/slice/slice.wgsl` — delinearizes the output index over
the contiguous output strides, maps the sliced-dim coordinate back to
the input (`start + coord*step`), relinearizes over the input strides.
@exported-using-ghexport

Differential Revision:
[D108793168](https://our.internmc.facebook.com/intern/diff/D108793168/)

Differential Revision:
[D108793168](https://our.internmc.facebook.com/intern/diff/D108793168)
JulianCloudNTH added a commit that referenced this pull request Jun 26, 2026
…work) (#20395)

Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0)
(oldest at bottom):
* #20465
* #20464
* #20463
* #20435
* #20399
* #20398
* #20397
* #20396
* __->__ #20395
* #20394
* #20393
* #20392
* #20391
* #20390
* #20363
* #20362
* #20361
* #20360
* #20359



Registers `aten.slice_copy.Tensor` in the `cases.py` op-test framework:
a `_slice_suite` of 4 configs (leading-dim slice `[:,1:5]`, last-dim
slice `[...,1:3]`, step-2 `[:,0:8:2]`, negative-end `[:,1:-1]`) that
`generate_op_tests` exports via `VulkanPartitioner` and compares to a
torch golden on Dawn. Also adds `test/ops/slice/test_slice.py`
(`SliceModule` + `CONFIGS` + export-delegation/eager smoke test) and the
`aten.slice_copy.Tensor` partitioner-allowlist entry in `tester.py`.
@exported-using-ghexport

Differential Revision:
[D108793151](https://our.internmc.facebook.com/intern/diff/D108793151/)

Differential Revision:
[D108793151](https://our.internmc.facebook.com/intern/diff/D108793151)
@JulianCloudNTH JulianCloudNTH merged commit 11567a3 into gh/JulianCloudNTH/45/base Jun 26, 2026
178 of 183 checks passed
@JulianCloudNTH JulianCloudNTH deleted the gh/JulianCloudNTH/45/head branch June 26, 2026 17:20
JulianCloudNTH added a commit that referenced this pull request Jun 26, 2026
…mework) (#20397)

Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0)
(oldest at bottom):
* #20465
* #20464
* #20463
* #20435
* #20399
* #20398
* __->__ #20397
* #20396
* #20395
* #20394
* #20393
* #20392
* #20391
* #20390
* #20363
* #20362
* #20361
* #20360
* #20359



Registers `aten.permute_copy.default` in the `cases.py` op-test
framework: a `_permute_suite` of 4 configs (3D rotation, 4D middle-dim
transpose, 2D transpose, full 4D shuffle) that `generate_op_tests`
exports via `VulkanPartitioner` and compares to a torch golden on Dawn.
Also adds `test/ops/permute/test_permute.py` (`PermuteModule` +
`CONFIGS` + `_op_delegated` smoke test) and the
`aten.permute_copy.default` partitioner-allowlist entry in `tester.py`.
@exported-using-ghexport

Differential Revision:
[D108793156](https://our.internmc.facebook.com/intern/diff/D108793156/)

Differential Revision:
[D108793156](https://our.internmc.facebook.com/intern/diff/D108793156)
JulianCloudNTH added a commit to JulianCloudNTH/executorch that referenced this pull request Jun 26, 2026
…ermute_copy.default)

Pull Request resolved: pytorch#20396

Adds `aten.permute_copy.default` (a coordinate-reorder gather) to the WebGPU delegate, and the `IntList` graph value type it needs to read its `dims` argument.

Composition:
- `runtime/WebGPUGraph.{h,cpp}` — adds `ValueType::IntList` backed by `std::vector<std::vector<int64_t>> int_lists_` + `get_int_list(int)`; `build()` deserializes `vkgraph::GraphTypes::IntList` via `value_as_IntList()->items()` (int64, matching the FlatBuffer `[long]`); mirrors the existing scalar value plumbing.
- `runtime/ops/permute/Permute.cpp` — reads the permutation via `get_int_list`, normalizes negative dims, validates it is a permutation of `[0, ndim)`, builds two `TensorMeta` UBOs + a `PermuteParams{perm: vec4<u32>}` uniform, guards fp32 + rank≤4, dispatches over `compute_1d_workgroup_count(out.numel)` with `override wg_size`; releases all uniforms after the bind group.
- `runtime/ops/permute/permute.wgsl` — delinearizes the output index over the contiguous output strides, reads `input` at `in.strides[perm[d]]` per dim (mirrors Vulkan `permute_buffer.glsl`).
- Registers both `aten.permute_copy.default` and `aten.permute.default` to the same handler.
ghstack-source-id: 397026548
@exported-using-ghexport

Differential Revision: [D108793162](https://our.internmc.facebook.com/intern/diff/D108793162/)
JulianCloudNTH added a commit that referenced this pull request Jun 26, 2026
### Summary
Manual merge of four WebGPU-delegate op PRs that landed internally but
could not auto-merge
to `main`. These are stacked ghstack PRs — when the lower PRs in the
stack merged, their head
branches were deleted and these four PRs' base branches were orphaned,
so the orig-PR
proposer failed with `422 base invalid`. This PR re-lands the same four
commits (identical
content to the originals, flat test layout) as a clean stack on top of
current `main`:

- [#20394](#20394) — Add
`slice_copy` op
  (`aten.slice_copy.Tensor`)
- [#20395](#20395) —
`slice_copy` op test suite
  (cases.py op-test framework)
- [#20396](#20396) — Add
`permute_copy` + `IntList`
  graph support (`aten.permute_copy.default`)
- [#20397](#20397) —
`permute_copy` op test suite
  (cases.py op-test framework)

  ### Test plan
Each op ships with its `cases.py` op-test suite (exported via
`VulkanPartitioner`, compared
to a torch golden on Dawn) plus an export-delegation smoke test,
exercised by the WebGPU
op-test CI (`etvk-*`). Verified internally; content is identical to the
original four PRs.
  
  @diff-train-skip-merge
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants